The present invention relates to television, including interactive television, animation, user telecommunications networks, and in particular to a method and system for creating video programs in a virtual studio, including game and animation programs with interactive interaction between the actors and objects of virtual reality.
Being a new technology, virtual studios gave birth to new terms and concepts.
A virtual studioxe2x80x94this is a new technology of filming people on a background of virtual decorations which are formed with the aid of computers. A virtual studio, as distinctive from conventional studios, is based on the use of images created by a computer. The elements of a virtual studio are in the memory of a computer in the form of models of virtual space. A virtual studio may be used for live, real time transmission conditions and for filming with subsequent processing. The whole complex of equipment for realization of this possibility or the virtual space itself in which a human is immersed is often called a virtual studio.
A blue room (chromakey room, blue box) is a usual television studio, all the walls and floor of which are painted with a uniform blue color. All filming of a user (actor, host) made with television cameras is made in this studio. The filming therein is made with several cameras, the cameras being used in turn, changing the point from which the shots are made. The image is sent to a computer. In the computer the image of the user is separated from the blue background and is superimposed with the decorations of the modeled virtual space at the angle of view of the virtual cameras (method of rear projection). In order to coordinate the actions of the host, certain marks may be made on the floor of the blue room with the arrangement of auxiliary objects: cubes, posts, etc. Such steps will help the host to at least approximately assess the geometry of the virtual space.
Virtual Setxe2x80x94this is an analogue of a three-dimensional stage in classical packets for computer graphics, comprising information on all virtual decorations, their animation and conversion in time, sounds, the number of virtual cameras, the set of textures and the live sources of the video image.
Virtual cameraxe2x80x94this is a camera in the virtual set, which is provided with the parameters of a television camera in a real studio. These parameters are supplied by sensors secured on the lens of a camera and determine the position, orientation and focus of the camera. As a result of this, the perspective at which a person is seen in the television camera will coincide with the perspective of the decorations for the virtual camera.
A ruler of a camera-computerxe2x80x94this is a part of the virtual studio complex, which includes a video camera with sensors, a computer with a unit containing the necessary software, and a unit aligning the image of an actor, being filmed in a chromakey room, with the virtual decorations. This autonomous chain makes it possible for an operator to constantly view an actor in the virtual set. The presence of several such chains makes it possible to mix the plans from different virtual cameras.
A method of filming television programs with the use of a virtual studio is known, wherein sensors, mounted on all of the video cameras, continuously transmit information on the position of each of the video cameras in space, the direction and focal distance of the lens. The obtained data are fed to the computer into the virtual space image calculation unit at the angle of view of the virtual camera (see Cyberset O, High-end Virtual Set, catalogue of the ORAD Inc. firm, 1997).
In the blue room of the virtual studio in which the actor is being filmed, he can only imitate interaction with objects of the virtual space. And only the operator sees the actor and virtual space together. The actor should remember the approximate positioning of the objects in virtual space, and during the filming he may only imitate interaction with the virtual objects. He may rehearse his movements and interaction with the objects of virtual reality beforehand in order to achieve the necessary matching during the filming.
A drawback of the known method is that real interaction with the objects of virtual space does not take place, since the actor himself does not see these objects. Interactive interaction of the objects of virtual space with the actor cannot be carried out either, since in the computer in which virtual space is modeled, there is no information on the actions of a live actor, on his position and orientation. The user (actor) may move only within the limits of the chromakey room. Without additional accessories he cannot imitate free movement over large distances, for example, along the corridors of infinite labyrinths.
A method of creating animated films is also known, wherein information on the movements of live beings, in particular, a human is used (see Motion Analysis, Animation Performance Capture for Film, Video and Games Production, catalogue of the Motion Analysis Corporation, 1996). Data on position and orientation are taken dynamically from a moving person with the aid of a system of sensors. These data are collected into a library of movements and are used in the animation of an image created by an artist. With modem powerful computers, such animation may be obtained in a real time condition. A person moving in an appropriate manner may observe on the screen of a computer that his computer image exactly repeats his real movements, and if there are special sensors, the mimics of the person. Then, with the aid of an editing program the episodes earlier created in this manner are combined with the episodes of animated cartoons into one whole. This is a lengthy and laborious procedure.
The object of the present invention is to create a method and system for creating video programs, in particular television and animated cartoon programs, which do not have the drawbacks indicated above. The technical result thereby achieved is the provision of interactive interaction of an actor with objects of virtual space and the natural movement of the actor in virtual space for any distances, in any direction, and also simplification and acceleration of the procedure of creating animation programs which could in the ideal case go on the air in real time condition, simultaneously with their moment of creation.
This result is achieved in that in a method for creating video programs, during the filming of a television program, wherein virtual space is modeled in which an actor should be displayed, the actor is filmed in a chromakey room of a studio, current parameters of a television camera are determined, an image of virtual space is formed at an angle of view of a virtual camera with parameters of a real television camera, an image of the actor is superimposed with the formed image of virtual space, in accordance with the invention, an angle of view of the actor is determined, an image of virtual space is formed which corresponds to the angle of view of the actor, and the formed image of virtual space is displayed to the actor.
Wherein it is preferable that interaction of the actor with objects of virtual space be carried out with the aid of a manipulator with six degrees of freedom, the position and orientation of the actor be tracked and the obtained data be used to make changes in the displayed virtual space, wherein during the filming the actor is provided with the possibility of imitating movement for any distances in any direction.
Furthermore, the filming of the actor and display to him of virtual space are carried out on the basis of the stroboscopic effect, wherein the filming and display of virtual space to the actor alternate with a predetermined frequency.
Furthermore, an image of the actor, filmed by a real camera, the parameters of the camera and data on changes in virtual space are preferably transmitted through a telecommunications network to a user device, where an image of virtual space is formed at the angle of view of the virtual camera with parameters of a real camera and the image of the actor received by the user device is superimposed with the formed image of virtual space, wherein the input of control commands, transmitted over the telecommunications network into the studio for corresponding change of virtual space, is carried out in the user device.
The indicated technical result is also achieved in that a system for creating video programs, mainly filming of television programs, comprising a chromakey room, television cameras provided with sensors of the parameters, a virtual space model unit, a unit forming an image of virtual space at an angle of view of a television camera connected thereto and an image superimposition unit, wherein outputs of the sensors of the parameters of a television camera are connected to a corresponding input of the unit forming an image of virtual space at the angle of view of the television camera, an output of the television camera is connected to a first input of the image superimposition unit, second and third inputs of which are connected respectively to an output of an alpha-mask and to a background output of the unit forming an image of virtual space at the angle of view of the television camera, and an output of the image superimposition unit is an output for the formed image signal, in accordance with the invention comprises a means for determining the angle of view of the actor, a unit forming an image of virtual space at the angle of view of the actor, which is coupled by two-way communication with the virtual space model unit, and a means for displaying virtual space to the actor, preferably made in the form of a virtual head-piece, wherein an output of the means for determining the angle of view of the actor is connected to a corresponding input of the unit forming virtual space at the angle of view of the actor, an output of which is connected to an input of the means for displaying virtual space to the actor.
Wherein it is preferable that it comprise a means for interactive interaction of an actor and objects of virtual space, and a means for determining the position and orientation of the actor, which are coupled to the virtual space model unit.
Furthermore, the system preferably comprises a channel of a telecommunications network, a connection unit coupled to the channel of the telecommunications network and to the virtual space model unit, and, at least, one user device comprising a user processor device coupled to the channel of the telecommunications network, a user unit for forming virtual space at the angle of view of the television camera, a user unit for superimposing the images, and a unit displaying the superimposed image, wherein a first output of the user processor device is connected to an input of the user unit for forming virtual space, a second output of the user processor device is connected to a first input of the user unit for superimposing the images, second and third inputs of which are connected to corresponding outputs of the user unit for forming virtual space, and an output is connected to an input of the unit displaying the superimposed image.
Wherein the system preferably also comprises a user control unit, an output of which is connected to a corresponding input of the user processor device, and a unit processing control commands of users, which is connected by two-way communication to the connection unit and the virtual space model unit.
The indicated technical result is achieved in that in a method for creating video programs, mainly animation programs, wherein a virtual space model is formed, data on the position and orientation of the actor are registered in dynamic conditions, objects of the virtual space are animated with the use of registered data on the position and orientation of the actor, in accordance with the invention, during registration of data on the position and orientation of the actor, the angle of view of the actor is determined, virtual space is formed and displayed at that angle of view, and registration of data on the position and orientation of the actor is carried out during his interaction with objects of virtual space.
Wherein, the actor is preferably arranged on a means for imitating movement for any distance in any direction.
Furthermore, data on changes in virtual space, including data being registered on the position and orientation of the actor and interaction with objects of the virtual space, are preferably transmitted through the telecommunications network to the user device to form an image of virtual space with animated objects and display them to the user, wherein the angle of view for forming an image of virtual space is preferably chosen by the user.
The indicated technical result is also achieved in that a system for creating video programs, mainly animated films, comprising an animation studio including a virtual space model unit, coupled thereto unit for forming an image of virtual space, a means for determining the position and orientation of the actor which is coupled to the virtual space model unit, in accordance with the invention comprises a means for determining the angle of view of the actor, a unit for forming virtual space at the angle of view of the actor, connected to the virtual space model unit, and a means for displaying virtual space to the actor, wherein an output of the means for determining the angle of view of the actor is connected to a corresponding input of the unit forming virtual space at the angle of view of the actor, an output of which is connected to an input of the means for displaying virtual space to the actor.
Wherein the system additionally comprises a channel of a telecommunications network, a connection unit coupled to the channel of the telecommunications network and to the virtual space model unit, and a user device comprising a user processor device coupled to the channel of the telecommunications network, a user unit for forming virtual space and a display unit, wherein an input of the user unit for forming virtual space is connected to an output of the user processor device, and its outputxe2x80x94to an input of the display unit.
Furthermore, the animation studio preferably also comprises a unit for processing control commands of the users, which is coupled to the connection unit and to the virtual space model unit, and the user device comprises a control unit, a first output of which is connected to an input of the user processing device, and a second outputxe2x80x94to a first input of the user unit for forming virtual space.